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Abstract 

Synchronization is known to play a vital role within many highly connected neural systems 
such as the olfactory systems of fish and insects. In this paper we show how one can robustly 
and effectively perform practical computations using small perturbations to a very simple globally 
coupled network of coupled oscillators. Computations are performed by exploiting the spatio- 
temporal dynamics of a robust attracting heteroclinic network (also referred to as 'winnerless 
competition' dynamics). We use different cluster synchronization states to encode memory states 
and use this to design a simple multi-base counter. The simulations indicate that this gives a 
robust computational system exploiting the natural dynamics of the system. 

PACS numbers: 05.45, 87.10,+c 
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Recent experimental and theoretical work by several authors has highlighted the crucial 
role of spatio-temporal dynamics in neural systems. In particular, work on olfactory systems 
of insects and fish has demonstrated that recognized odors manifest themselves as different 
spatio-temporal outputs from certain sub-systems such as for insect antennal 

lobes (AL) and 0, Q for the Zebrafish olfactory bulb (OB). It has been suggested that the 
dynamics of the network responsible for the transformation of input to output of the AL / OB 
for these systems is one of 'winnerless competition' ; this is a class of attracting; 

dynamics consisting of saddle periodic orbits linked together by unstable manifolds. Such 
attractors may seem at first unnatural but in systems that are close to symmetric and in 
systems that have invariant subspaces they may be robust; i.e. they may persist under 
arbitrary perturbations that preserve the symmetric (or invariant subspace) structure. Such 



attractors have been found in a range of physically relevant models and experiments |lCtlll|. 

The AL/OB system is far from beinga simple relay; it shows effects such as short term 
memory, anticipation and computation p|. The aim of this paper is to show that globally 
coupled systems idealizing the AL/OB architecture can be used to explicitly design compu- 
tational systems by exploiting robust heteroclinic attractors (winnerless competition). We 



use globally coup 
cluster states 



ed phase oscillators in regimes where one finds slow switching between 
Q 13) Previous work has shown 16] that robust heteroclinic attrac- 
tors for a particular system of five globally coupled phase oscillators allow one to encode 
up to twenty different memory states as synchronized clusters, and moreover to effectively 
move between them by means of small perturbations. The number of such states scales 
exponentially with iV the number of oscillators. In this paper we go further to show that 
one can in principle perform arbitrary computations using a globally coupled cluster for 
'memory/timing' on addition of perturbations that may have v ery low amplitude. 



We consider perturbations to the model of Hansel et al. 
oscillators with phases 8i G T = [0, 2ir) given by 

N 



12j for N globally coupled 



^ = u + ^J2a(0i - 0j) + VWi(t) + eli(t), (1) 



N 

3=1 



when i — 1, • • • ,N and g(<p) = — sin(0 + a) + r sin (20); see also [13, [l4|, [l5|. The quantities 
Wi(t) represent derivatives of independent, identically distributed Brownian processes with 
zero mean and unit variance per unit time. The inputs ]/«(£) | < 1 are used to control the 
state of the system. We include rj and e, small parameters that control the strength of the 



noise and perturbations respectively. 

The system (JTJ) for the unperturbed system (r) — e — 0) has symmetry Sat correspond- 
ing to permutations of the N oscillators. Using these symmetries one can identify robust 
heteroclinic attractors (first noted theoretically in and then practically for this example 
in Q|) that exist for open sets of parameters. For certain parameter values the attractors 
consist of saddle periodic orbits of symmetry x Sx-k, i-e. states where all oscillators are 
in one of two possible phases at any point in time and such that precisely k of them are in 
one phase. The system also has an S 1 phase-shift symmetry that allows one to characterize 
periodic orbits as group orbits. For 

N = 5, a = 1.25, u = 5, r = 0.25, (2) 

the only attractor for unperturbed (JTJ consists of twenty periodic orbits with symmetry 
S2 x S3 and their unstable invariant manifolds. This forms them into a single strongly 
connected network. 

The periodic orbits involved are S 1 orbits of Xj = (0, 0, 0, ipj, ipj) with j = 1,2 and 
permutations thereof. We list a set of permutations with k = 1, • • • , 10 that map the 
periodic orbit through Xj onto the 10 possible symmetric images and denote by Pj° the 
periodic orbit given by the S 1 orbit of o^Xj for j = 1, 2 and k = 1, • • • , 10. These periodic 
orbits are listed in Table HI For the given parameters (j2J), i>\ = 1.339 and ^2 = 0.799 jlfil ]. 
If we write £ = Uj=i W u (Pj) where W U (P) is the unstable manifold of the saddle 
periodic orbit P then any randomly chosen initial condition approaches S; this manifests 
itself as a sequence of 'slowing down' switchings between different cluster states P*. Figure ^ 
shows a detail of the unstable manifold of one of the Pf. 

For larger N displays heteroclinic cycles between x S^^k symmetry states for a 
variety of k with iV/3 < k < N/2 [14| : this gives rise to more complicated attracting het- 
eroclinic networks that are currently under investigation by the authors. The unperturbed 
dynamics is such that after a transient period the system state is near one of the except 
for increasingly occasional short transitions. As £ is asymptotically stable this is also true 
even for the perturbed system as long as the perturbation in small enough. We exploit this 
feature for our computational system. Typical initial conditions for the unperturbed system 
lead to an asymptotic state that is a slowing-down oscillation between P* and P 2 fc for some 
k determined by the initial condition. 
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Trajectories of the perturbed system may visit many or all P* where j always alternates 
between 1 and 2 and k changes on a longer timescale [l^j]. This is because the unstable 
manifold of Pf is two dimensional and only exceptional trajectories on this connect to P\ 
with k ^ /. In the presence of noise we refer to the average time of switching between 

n 

Pj as the cycling time T c ; one can show that this scales as — In 77 [12]. The average time 
of switching between P- with different k we refer to as the switching time T s . The order 



parameter \ = i Ylk=i e%9k °ften considered for such systems [12. 1141 oscillates at a rate 
given by the cycling time. Because this measure is invariant under permutations of the 
phases it is not possible to detect switches and compute T s using only x- 

One can view the globally coupled system ([TJ as a computational system with five inputs 
Ii(t), i = 1..5 and twenty outputs Yj ! (9) = \9 — P^|i (where = Yli I^D' namely if 
is small then the trajectory is close to P- . Figure [21 shows the statistics of 5, the local 
minimum of the Y^ over one cycle. The mean (5) varies proportionally as the square root 
of the noise, reminiscent of the algebraic scaling in heteroclinic switching rates with noise 
level found in ^?]]. Defining the mean probability of switching per cycle P s , we can estimate 
P s = T c /T s . Numerical simulations indicate that T s /T c = 0(r7 1//2 ) as 77 — > 0; see Figure 01 
where we detect close approaches by \9 — P- \i < A where A = lOy/e. This ensures that 
all the switches found in Figure El are registered while it avoids false positive detections of 
switching, as (for instance) may occur on a trajectories moving between Pi and P 2 X passing 
very close to P 2 9 or P 2 2 (see Figure (JI})). One can then use simple logic gates to connect up 
outputs and inputs to form a computational system. 

As a proof-of-principal, we have used (0) to construct a counter that counts in either base 
5 or base 2 depending on initial condition to the phases. The inputs we consider are C (t) a 
clock pulse that occurs at approximately regular intervals and X(t) an input that may or may 
not occur between clock pulses. We construct the system so that pulses of C(t) occurring 
at a state P 2 fc perturb the system along the unstable manifold towards P x and pulses of 
X(t) occurring at a state Pf perturb the system along the unstable manifold towards P^ 
where £ = [1, 3, 4, 5, 6, 2, 8, 7, 9, 10]. In each case this corresponds to simply applying a pulse 
to input ip(fc) where p = [4, 4, 1, 3, 2, 5, 3, 3, 2, 1]. In this way we effectively define functions 
Jj = Fi{Y^)X + F 2 (Y J k )C that steer the state of the system around the network rapidly and 
reliably even when e is very small, depending on the inputs X and C. We show the setup 
schematically in Figure EJ Figure El shows the counter functioning accurately for an initial 



tcninq 1 

HQ 
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condition near P\. Figure |H1 shows simulations on changing parameters and in the last case, 
initial conditions. 

There is a trade-off between on the one hand the speed of computation and on the other 
hand its accuracy and sensitivity to noise. In particular the input amplitude e gives a 
characteristic cycle time as does the noise amplitude 77. For effective computation in the 
above system we need 1 ^> e ^> rj and the clock cycles must lie between the cycle times 
associated to the two perturbation amplitudes. 

Many models use oscillators to perform computational tasks, for example The 
novelty in this work is that we demonstrate how one of the simplest possible types of coupling 
can give rise to robust attracting heteroclinic networks that one can exploit to perform 
computations. As in winnerless competition models DDD the average phase differences in 
the globally coupled system change very little depending on the state of the system and the 
computation is performed by following natural trajectories within the system. Moreover, the 
state of the system is not detectable from individual or 'mean' cell outputs but is encoded 
globally. By contrast, if we use asymptotically stable attractors for storing memory states 
comparatively large perturbations to the system will be needed to lift it out of the basin of 
attraction into another state. 

The mechanism we consider is a winnerless competition model using (a) two dimensional 
manifolds which allow many possibilities to be considered at each state and so allows the 
network to scale to large numbers of states without greatly increasing the average length of 
path (b) the dynamics requires no careful tuning for a range of coupling parameters, nor 
plasticity in the coupling. We use two populations of processing elements (a) the globally 
coupled oscillators 9i have the function of a 'memory/timing' circuit and (b) observables 
from the system that detect presence of a particular cluster state. The latter are analogous 
to certain Kenvon Cells in the mushroom body that receive inputs from an insect antennal 



lobe 



DDD 



18j | and decode a large dense set of states in a relatively low dimensional space 



to a sparse set of states in a higher dimensional space. 

We believe that the simple global coupling used within the 'processing element' of our 
model is a positive feature. Although symmetry is never achieved, the observation that the 
heteroclinic attractor is asymptotically stable means that the dynamics are well modelled 
even when global coupling is perturbed. There may be evolutionary advantages to having 
neural subsystems that attempt to approach symmetric coupling as this can presumably 
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be encoded genetically very simply (see [20] for other approaches to neural information 
processing that use symmetries). The fact that the dynamics of the unperturbed system is 
robust means that heteroclinic attractors are model-independent and will exist in globally 



coupled systems of phase oscillators 
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21IL coupled ODE oscillators or even piecewise 



>rs Q- 



smooth systems of phase-resetting oscillators 

The dynamics we use is not present in N < 4 oscillators; to find robust heteroclinic 
attractors with unstable manifolds of higher than one dimension we need N > 5 0- The 
ideas presented should scale well on increasing N beyond 5; the number of cluster states 
grows exponentially but the diameter of the heteroclinic network need not grow, meaning 
that rapid transition from one state to another is still possible. Note we do not claim that 
any particular AL/OB neural system functions via perturbations to a robust heteroclinic 
attractor (though this has been suggested If). We do show that a globally coupled system 
of phase oscillators allows rapid and reliable computation. 
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1 


(O,O,O,-0j,^j) 


p2 p9 plO 
r 1t r %1 r 2 


2 


(^•,^•,0,0,0) 


pi p3 p6 
r 1i r 1i r 2 


3 


(0,0,^,0,^) 


p2 p4 p8 
r 2i r 2i ^2 


4 


(o,^,o,Vi,o) 


p3 p5 plO 
r 2t r 2^ r 2 


5 


(^•,0,0,0,^) 


pi p6 p9 
■^2 i r 2 ' ^2 


6 


(0,0,^,^,0) 


p2 p5 p7 
r 2 i r 2 ) ^2 


7 


(0,^,0,0,^) 


p6 p8 plO 
■^2 > r 2 ) r 2 


8 


(^,0,0,^,0) 


p3 p7 p9 
r 2) r 2! ^2 


9 


(0,^,^,0,0) 


pi p5 p8 
r 2 ' ""2 > ^2 


10 


(^,0,^,0,0) 


pi p4 p7 
r 1i r 1i r 2 



TABLE I: Representative points on the periodic orbits Pj? (j = 1,2, k = 1, ••• ,10) in the 
unperturbed heteroclinic network (jTjl; parameters as in @. In all cases W u {Pf) D P£_j, while 
W u (Pi) contains the addition points listed (A denotes the closure of the set A). 




FIG. 1: Schematic showing part of the two dimensional unstable manifold W u (Pl); almost all 
trajectories leaving from converge to P\ but exceptional trajectories converge to P\ for three 
possible I ^ 1, two of which are shown here. 
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FIG. 2: Dots show the value of 5 = inf{Y^(t) : k,j, t G [to, to + T c )}, the smallest value of any 
Yj° during a cycle time T c in the presence of noise with amplitude rj, after transients have decayed 
and for a range of random initial conditions. The mean closest approach (taking 100 cycles) scales 
as (6) = 0(y/rj). 
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FIG. 3: Probability of switching P s at each cycle for specified 77 with average over 100 trials. Error 
bars show two standard deviations and a least squares fit line is also included. This fits P s ~ y/rj. 
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Output 




FIG. 4: Schematic wiring diagram for using the five-oscillator system for finite-state computation. 
The logic circuit steers the states around two possible periodic sequences of Pj depending on initial 
condition. The input to the system X and clock C are connected to one of the Ii depending only 
on which of the Yj* is lowest. 
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FIG. 5: Simulation of the counter for C(t) and X(t) as shown in the top graphs, for r\ = 10 
and e = 10 -10 . The oscillator phase differences sin(#i — 9j) are shown, and the state index (bottom 
graph) cycles one step through states where k G {2,3,4,5,6} each time an input on X is 
received. The same output is obtained robustly for a range of parameters. 
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FIG. 6: Index giving the state of the counter for (a) the same inputs and initial condition as 
Figure [21 (b) as for (a) but increased rj (observe the appearance of an erroneous switch just before 
t = 900); (c) as for (a) also reliably functions with the clock rate slowed from 50 to 70 time units; 
(d) as for (a) but starting with an initial condition near state 7; this shows the presence of a second 
cycle in the network. 
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